In general, a computer can display a plurality of resolutions in correspondence with that of a display device. For example, if the maximum resolution of the display device is XGA, the computer is often required to display images of the XGA resolution or lower (VGA, SVGA). This is to allow the user to select a desired resolution to display more information on a screen or to display in a larger scale so that he or she can view an image more easily.
In this case, a resolution conversion technique is required for the computer main body to provide a desired resolution to the display device.
As for televisions, an NTSC scheme is adopted in Japan. In NTSC, the number of scan lines ranges from 483 to 485, the number of pixels per scan line ranges from 720 to 760, and the total number of pixels is about 350,000. Since display is normally attained by interlaced scans, the practical resolution is about 150,000 pixels.
By contrast, in recent years, a Hi-Vision scheme has prevailed. The Hi-Vision scheme is also called HD, and assures a resolution of about 750,000 pixels. The conventional NTSC broadcast is called SD in correspondence with HD.
Furthermore, in digital broadcast, image formats such as 480i, 720p, and 1080i are available. 480i indicates 640×480 interlaced scans, 720p indicates 1,024×720 progressive scans, and 1080i indicates 1,280×1,080 interlaced scans.
In addition, there are many applications such as image display of personal computers, video display of games and digital cameras, and multi-frame display for displaying a plurality of frames, and the chances of displaying many image formats on a television are increasing.
Note that analog data of the conventional NTSC broadcast can relatively easily undergo image format conversion by changing the sampling rate. However, in order to attain format conversion of digital data such as digital broadcast, digital camera images, and the like, image data must undergo interpolation and decimation.
Also, in order to provide a display effect such as multi-frame display, resolution conversion for enlarging/reducing an image is required together with the image format conversion.
In general, a large frame memory is required to execute such resolution conversion. In order to execute resolution conversion without any frame memory, all processes must be done within 1V (one vertical period) of an input image. If the processing time exceeds 1V, the next frame process is influenced, and a large memory is consequently required. An example of resolution conversion which is made without using any frame memory is proposed by, e.g., “Scan Converter” (Japanese Patent Laid-Open No. 11-338408) and the like. However, the scheme proposed by this reference focuses on image formation conversion, but does not aim at achievement of higher image quality when the user makes desired resolution conversion.
In order to aim at achievement of higher image quality, a method of referring to a plurality of pixels is generally used. Interpolation by referring to only a single pixel is merely prediction from that pixel. However, when a plurality of pixels are used, changes in pixels before and after the pixel position to be interpolated can be used, and the reliability of prediction can be improved. If two reference pixels are used, a change in pixel at that time can be expressed by a line; if three or more reference pixels are used, it can be expressed by a curve.
As such interpolation method that refers to a plurality of pixels, linear interpolation, three-dimensional (3D) convolution, and the like are known. Linear interpolation interpolates a pixel using two pixels (4 pixels in a matrix), and 3D convolution interpolates a pixel from four pixels (16 pixels in a matrix). In general, the image quality becomes higher but arithmetic processes become more complicated with increasing number of reference pixels.
The convolution arithmetic operation will be explained below. The convolution arithmetic operation uses 16 pixels in a source image and eight coefficients per output pixel.
If we have:
pre-conversion coordinates: x, y
post-conversion coordinates: X, Yconversion formulas: X=A*x, Y=B*y (A: x-scale, B: y-scale)since the pre-conversion coordinates (x, y) are discontinuous numbers and the post-conversion coordinates (X, Y) are continuous numbers in practice, the pre-conversion coordinates are obtained using inverse conversion formulas:x=X/Ay=Y/B
The integral part of each inverse conversion result is used to derive the pixel value of the source image as a pre-conversion coordinate. The decimal part (remainder) is used to generate a coefficient.
For each pixel (X, Y), 16 source image pixel values G are acquired from source image coordinates (x, y).
row y − 1G11G21G31G41row yG12G22G32G42row y + 1G13G23G33G43row y + 2G14G24G34G44column x − 1column xcolumn x + 1column x + 2
Let x−[x] and y−[y] be the decimal parts of values obtained by inverse conversion. Then, coefficients f(t) are calculated by:
                              x1          =                      1            +                          (                              x                -                                  [                  x                  ]                                            )                                                            y1          =                      1            +                          (                              y                -                                  y                  ⁡                                      [                    y                    ]                                                              )                                                                    x2          =                      (                          x              -                              [                x                ]                                      )                                                y2          =                      (                          y              -                              [                y                ]                                      )                                                        x3          =                      1            -                          (                              x                -                                  [                  x                  ]                                            )                                                            y3          =                      1            -                          (                              y                -                                  [                  y                  ]                                            )                                                                    x4          =                      2            -                          (                              x                -                                  [                  x                  ]                                            )                                                            y4          =                      2            -                          (                              y                -                                  [                  y                  ]                                            )                                                                                    f            ⁡                          (              t              )                                ≈                      (                                                                                1                    -                                          2                      ⁢                                                                                                  t                                                                          2                                                              +                                                                                          t                                                                    3                                                                                                                                        4                    -                                          8                      ⁢                                                                      t                                                                                      +                                          5                      ⁢                                                                                                  t                                                                          2                                                              -                                                                                          t                                                                    3                                                                                                                    0                                                                                                                                    (                                  0                  ≤                                                          t                                                        <                  1                                )                                                                                        (                                  1                  ≤                                                          t                                                        <                  2                                )                                                                                        (                                  2                  ≤                                                          t                                                                      )                                                        
From the above 16 pixel values (G11 to G44) of the source image and eight coefficients (x1 to x4i, y1 to y4), a pixel value P of an image after conversion is calculated by:
  P  =            (                        f          ⁡                      (                          y              1                        )                          ⁢                  f          ⁡                      (                          y              2                        )                          ⁢                  f          ⁡                      (                          y              3                        )                          ⁢                  f          ⁡                      (                          y              4                        )                              )        ⁢          (                                                  G              11                                                          G              21                                                          G              31                                                          G              41                                                                          G              12                                                          G              22                                                          G              32                                                          G              42                                                                          G              13                                                          G              23                                                          G              33                                                          G              43                                                                          G              14                                                          G              24                                                          G              34                                                          G              44                                          )        ⁢          (                                                  f              ⁡                              (                                  x                  1                                )                                                                                        f              ⁡                              (                                  x                  2                                )                                                                                        f              ⁡                              (                                  x                  3                                )                                                                                        f              ⁡                              (                                  x                  4                                )                                                        )      
Note that 3D arithmetic operations are required to calculate the coefficients. In order to reduce this calculation volume, conditional selection may be made from the first x−[x] and y−[y] values, and fixed coefficients may be selected from a coefficient table, which is calculated in advance.
As described above, the convolution arithmetic operation requires 16 pixel values of the source image per output pixel. A method of reading out these 16 pixel values changes depending on scales, and identical pixels are often multiplied by different coefficients. For this reason, a method of sequentially processing input pixels cannot be used. Hence, a frame memory or a plurality of line buffers are required.
A method of making such 3D convolution arithmetic operations without using any frame memory is also proposed by “Display Control Apparatus and Method, and Display Apparatus” (Japanese Patent Laid-Open No. 8-297477). This reference has proposed a method of implementing resolution conversion based on 3D convolution arithmetic operations using only line buffers by providing a plurality of arithmetic processors.
However, since the convolution arithmetic operations require many multiplication processes and the circuit is complicated, the apparatus of Japanese Patent Laid-Open No. 8-297477 which comprises a plurality of arithmetic processors requires a large circuit scale.
When arithmetic operations are made using a single arithmetic processor without any frame memory, resolution conversion up to twice in the vertical and horizontal directions can be attained if arithmetic processes are executed using clocks about four times of input pixel clocks. However, in case of the convolution arithmetic operations, 4 (vertical)×4 (horizontal) pixels must be read per pixel. Pixels in the vertical direction can be simultaneously read out as long as a required number of line buffers are prepared. However, one pixel (column) can only be read out per clock in the horizontal direction.
In the above situation, it is demanded to make a single arithmetic circuit execute a plurality of different types of resolution conversion using line buffers.
Also, it is especially demanded to make a single arithmetic circuit execute a plurality of different types of resolution conversion based on 3D convolution arithmetic operations using line buffers.